Performance Metrics Cheat Sheet




\(\begin{aligned} {\text{Prevalence}} = \frac{\text{TP + FN}}{\text{TP + FP + TN + FN}} \end{aligned}\)


\(\begin{aligned} {\text{PPCR (Predicted Positives Condition Rate)}} = \frac{\text{TP + FP}}{\text{TP + FP + TN + FN}} \end{aligned}\)


\(\begin{aligned} \text{Sensitivity (Recall, True Positive Rate)} = \frac{\text{TP}}{\text{TP + FN}} = \frac{\text{TP}}{\text{Real Positives}} = \text{Prob( Predicted Positive | Real Positive )} \end{aligned}\)


\(\begin{aligned} \text{Specificity (True Negative Rate)} = \frac{\text{TN}}{\text{TN + FP}} = \frac{\text{TN}}{\text{Real Negatives}} = \text{Prob( Predicted Negative | Real Negative )} \end{aligned}\)


\(\begin{aligned} \text{PPV (Precision)} = \frac{\text{TP}}{\text{TP + FP}} = \frac{\text{TP}}{\text{Predicted Positives}} = \text{Prob( Real Positive | Predicted Positive )} \end{aligned}\)


\(\begin{aligned} \text{NPV} = \frac{\text{TN}}{\text{TN + FN}} = \frac{\text{TN}}{\text{Predicted Negatives}} = \text{Prob( Real Negative | Predicted Negative )} \end{aligned}\)


\(\begin{aligned} \text{Lift} = \frac{\text{PPV}}{\text{Prevalence}} = \frac{\cfrac{\text{TP}}{\text{TP + FP}}}{\cfrac{\text{TP + FN}}{\text{TP + FP + TN + FN}}} \end{aligned}\)


\(\begin{aligned} \text{Net Benefit} = \frac{\text{TP}}{\text{TP + FP + TN + FN}} - \frac{\text{FP}}{\text{TP + FP + TN + FN}} * {\frac{{p_{t}}}{{1 - p_{t}}}} \end{aligned}\)




Calibration

Smooth

## # A tibble: 10 x 4
##    model   quintile    phaty     phatx
##    <fct>      <int>    <dbl>     <dbl>
##  1 model 1        1 0        0.0000178
##  2 model 1        2 0.000626 0.000104 
##  3 model 1        3 0.00125  0.000330 
##  4 model 1        4 0.000626 0.000849 
##  5 model 1        5 0.00188  0.00202  
##  6 model 1        6 0.00501  0.00462  
##  7 model 1        7 0.00939  0.0108   
##  8 model 1        8 0.0157   0.0270   
##  9 model 1        9 0.0307   0.0841   
## 10 model 1       10 0.470    0.455    
## [1] -0.02348153  0.49311209

Discrete

## # A tibble: 10 x 4
##    model   quintile    phaty     phatx
##    <fct>      <int>    <dbl>     <dbl>
##  1 model 1        1 0        0.0000178
##  2 model 1        2 0.000626 0.000104 
##  3 model 1        3 0.00125  0.000330 
##  4 model 1        4 0.000626 0.000849 
##  5 model 1        5 0.00188  0.00202  
##  6 model 1        6 0.00501  0.00462  
##  7 model 1        7 0.00939  0.0108   
##  8 model 1        8 0.0157   0.0270   
##  9 model 1        9 0.0307   0.0841   
## 10 model 1       10 0.470    0.455    
## [1] -0.02348153  0.49311209

Probability Threshold Dependent Performance Metrics

By Probability Threshold

Performance Metrics Curves

ROC

Lift

Precision Recall

Gains

Decision

Performance Table

By Predicted Positives Condition Rate (PPCR)

Performance Metrics Curves

ROC

Lift

Precision Recall

Gains

Decision

Performance Table